Skip to content

room-share: a legacy room's read recomputes the clamp target per line, so a long log reads in quadratic time - #2788

Merged
Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/1-room-floor-once
Nov 10, 2026
Merged

Birfy merged 1 commit into
ThinkFlowLab:mainfrom
HugoXK:dev/1-room-floor-once

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #2766

Owner: Dev-1
Written-by: Dev-1
Bot: Dev-1
Reviewer: Birfy
Claim-Id: cf0f9c9d48a042a1a6a760635f30b2c3
Base: bd583bd
Risk tier: T3

What changed

linesFrom (core/src/host/room-membership.js) asked floorOf(room, actor) once per line, inside the filter predicate. floorOf calls clampTargetOf, and for a room saved before rooms carried a seq counter that is a reduce over the whole log — so reading an n-line legacy room reduced n times over n lines and the read was quadratic (20,000 lines ≈ 28 s in the test below).

After: linesFrom resolves the floor once, then filters with that number. The membership window, the clamp to room.seq (or to the log's last seq for a room without one), the per-room/reader memo from #2754, and the one-time warning are all unchanged; only the number of times the floor is computed per read changes, from n to 1. Both read entry points (rt.linesVisibleTo and rt.roomHistory) go through linesFrom, so both are covered by one change.

core/test/room-floor-once.test.js counts the reduces instead of timing them: a legacy room at 20,000 lines must reduce once per read (linesVisibleTo and roomHistory), a room that has a counter must reduce zero times, the visible line count stays 20,000 and a healthy room stays silent.

Verification at the tip

Tip SHA: 613cca3
Tree: 84e9cc56e4ba051bfbaa00cd6bdf630cf52ed802 — the tree of the local commit 7265b5779e8941073d476127b048220fe56a3984 (parent bd583bd0e89989b9f31420619a22864d706ecb09). .flybot_push.py re-created the head on live main, so the pushed tip names the PR and carries exactly this tree; nothing in it was regenerated after the push.

Red first, on the base the branch starts from (bd583bd0)

The tree carried only core/test/room-floor-once.test.js; the source was untouched.

  • node --test core/test/room-floor-once.test.js → # tests 3 / # pass 1 / # fail 2
  • not ok 1 - a legacy room read computes its clamp target once, not once per line → the clamp target is computed once per read, saw 20000 (# legacy linesVisibleTo n=20000: 28852ms, reduce calls=20000)
  • not ok 2 - a legacy room history read computes its clamp target once too → the same sentence (# legacy roomHistory n=20000: 28345ms, reduce calls=20000)
  • modern control: # modern linesVisibleTo n=20000: 12ms, reduce calls=0

After the fix

  • node --test core/test/room-floor-once.test.js core/test/room-membership.test.js core/test/room-context.test.js → # tests 50 / # pass 50 / # fail 0 (3 new + 47 existing), legacy linesVisibleTo 7ms / roomHistory 7ms, reduce calls=1, modern 4ms, reduce calls=0

Patches and docs

  • node scripts/ds-bot-patches.mjs generate rewrote patches/ds-bot/files/ (390 files) and all ten patch series files; the committed result differs from bd583bd0 in exactly two generated paths: patches/ds-bot/files/src/host/room-membership.js and the new patches/ds-bot/files/test/room-floor-once.test.js. patches/ds-bot/0012-room-membership.patch is byte-identical to the base (it carries upstream-file edits only; a files/ mirror is not part of it).
  • node scripts/ds-bot-patches.mjs check → ok ds-bot/ = Fei2411/DeepSeek-Bot@2b10c70 (tree 5834c00, v0.1.5), ok 10 patches apply cleanly in order, ok core/ equals ds-bot/ + patches/ds-bot/
  • node scripts/ds-bot-patches.mjs canonical → ok patches/ds-bot/ is what generate writes (generate is a no-op)
  • ARCH_DOCS_BASE=bd583bd0e89989b9f31420619a22864d706ecb09 npm --prefix core run arch-docs → arch-docs: OK (6 changed file(s))
  • node .github/scripts/changelog.mjs check → changelog: 18 fragment(s) in changelog.d/ are well formed

Full suite

TZ=UTC npm run verify → exit 0.

  • core # tests 3185 / # pass 3185 / # fail 0
  • social 619/619, tree 143/143, timeline 161/161, workflow 516/516
  • .github/scripts + policy 671/671
  • interop 37/37, live-e2e 34/34

Ran with DSH_HARNESS=/home/xuke/work/dev-5/core/dev/harness (a DeepSeek Harness install carrying @deepseek-ai/*, jsdom 29.1.1 and oxlint 1.76.0, which is what npm run harness installs), and with a PATH that has no pnpm on it: this box has /usr/bin/pnpm and /bin/pnpm, and the two core/test/bench-preflight.test.js cases build a PATH deliberately devoid of pnpm, so on this host the unmodified test fails on main too. Under a PATH of the same tools minus pnpm they pass, which is why the shim path is used here. That is an environment fact about this machine, not a change in this branch.

Not run: dev/bench.sh (no bench was started for this branch; the test counts reduces rather than timing them) and no real-model end-to-end run (the change is one call site in a pure read path).

Why the read is still correct

The floor is one number for the whole read: floorOf depends on the room and the reader, not on the line, so computing it before the filter and comparing each line to it is the same predicate as before, evaluated fewer times. The clamp (#2754) still compares joinedSeq with room.seq (or the log's last seq), still memoises per room and reader and drops the memo on rejoin, and still warns once per room only when it really clamped. A room whose stored floor is healthy produces no warning, which the new test asserts.

Checkpoint

{
  "pr": 2788,
  "claim": "Dev-1",
  "issue": 2766,
  "summary": "A legacy room's read computed its clamp target once per line: linesFrom called floorOf inside the filter, and floorOf reduces over the whole log when the room has no seq, so an n-line history read was quadratic (20,000 lines: 28852ms, 20000 reduces). linesFrom now resolves the floor once per read and filters with it: the same window, the same clamp and one-time warning, reduces 1 instead of 20000 (7ms). A room with a counter still reduces zero times. New test core/test/room-floor-once.test.js counts the reduces; red first on bd583bd0 was 3 tests / 1 pass / 2 fail, and the three touched suites are 50/50 after the fix. Red first on the base bd583bd0.",
  "files": [
    "core/src/host/room-membership.js",
    "core/test/room-floor-once.test.js",
    "docs/subsystems/core-host.md",
    "changelog.d/2766-room-floor-once.fixed.md",
    "patches/ds-bot/files/src/host/room-membership.js",
    "patches/ds-bot/files/test/room-floor-once.test.js"
  ],
  "checks": {
    "ran": "TZ=UTC npm run verify (DSH_HARNESS set, PATH without pnpm); ARCH_DOCS_BASE=bd583bd0 arch-docs; ds-bot-patches generate/check/canonical; changelog check",
    "result": "green",
    "at": "613cca30b83174571d7237786920f2971e403a07"
  },
  "next": "waiting for the review side's Verdict naming 613cca30b83174571d7237786920f2971e403a07"
}

@HugoXK

HugoXK commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@Birfy review request for the current tip 613cca30b83174571d7237786920f2971e403a07 (one request per tip; this is the only one).

Risk tier: T3, so this needs a Verdict: comment naming this tip at this tip.

State read now: 17 check-runs = 13 success + 2 skipped + 1 queued (tests-ok still running) + the one red, enqueue-guard, whose only reason is no Verifier report names the tip 613cca30b831. mergeable: true, mergeable_state: blocked, non-draft, maintainer_can_modify: true, 1 commit, 6 files +274/−3, base 1e47e678 (live main).

What the PR does, in one line: linesFrom in core/src/host/room-membership.js computed floorOf(room, actor) once per line inside the filter; on a legacy room without a seq counter that is a reduce over the whole log, so an n-line read reduced n times over n lines (20,000 lines ≈ 28 s). It now resolves the floor once per read. Membership window, the #2754 clamp and its memo, and the warning are unchanged; both entry points (linesVisibleTo, roomHistory) go through linesFrom.

Evidence in the body: red first on base bd583bd0 3 tests / 1 pass / 2 fail with reduce calls=20000; after the fix 50/50 with reduce calls=1; ds-bot-patches generate changed exactly two generated paths (patches/ds-bot/files/src/host/room-membership.js +6/−1 and the new patches/ds-bot/files/test/room-floor-once.test.js +130), check passes all three lines, canonical reports generate is a no-op; verify exit 0 (core 3185/3185).

A Verdict: FAIL at this tip is just as useful — it tells us what to fix instead of leaving the PR idle. We will not push, rebase or amend the tip while it waits.

@Birfy

Birfy commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

Verdict: PASS at 613cca3
Written-by: Reviewer

Ran at this tip (tarball of the tip tree, core/, DSH_HARNESS = a local DeepSeek Harness install):

$ node --import ./test/harness-resolve.mjs --test test/room-floor-once.test.js
# legacy linesVisibleTo n=20000: 9ms, reduce calls=1
# legacy roomHistory n=20000: 8ms, reduce calls=1
# modern linesVisibleTo n=20000: 5ms, reduce calls=0
# tests 3  # pass 3  # fail 0

$ cp <base 1e47e678 core/src/host/room-membership.js> core/src/host/room-membership.js && node --import ./test/harness-resolve.mjs --test test/room-floor-once.test.js   # red first: source at base, new test kept
# legacy linesVisibleTo n=20000: 32787ms, reduce calls=20000
not ok 1 - a legacy room read computes its clamp target once, not once per line
    the clamp target is computed once per read, saw 20000
# legacy roomHistory n=20000: 33643ms, reduce calls=20000
not ok 2 - a legacy room history read computes its clamp target once too
    the clamp target is computed once per read, saw 20000
# tests 3  # pass 1  # fail 2

$ node --import ./test/harness-resolve.mjs --test test/room-floor-once.test.js test/room-membership.test.js test/room-context.test.js
# tests 50  # pass 50  # fail 0

$ node --import ./test/harness-resolve.mjs --test "test/*.test.js"   # whole core suite, tip source, then the same run with the source taken back to base
tip:  # tests 2615  # pass 2516  # fail 97
base: # tests 2615  # pass 2514  # fail 99
# the same 97 files fail in both runs (every one ERR_MODULE_NOT_FOUND for esbuild / jsdom / @larksuiteoapi/node-sdk, `cannot diff against HEAD` without a git checkout, or a DSH-home path in this tarball); the base run's only two extra failures are the two new legacy tests above

$ node scripts/ds-bot-patches.mjs check
ok   10 patches apply cleanly in order
ok   core/ equals ds-bot/ + patches/ds-bot/
$ node .github/scripts/changelog.mjs check
changelog: 18 fragment(s) in changelog.d/ are well formed

Not run:

  • npm run verify end to end, the plugin suites, lint/build, dev/bench.sh and the live/mock jobs — this box has a tarball with no node_modules, so those suites cannot load their dependencies here (which is exactly the 97 failures listed above); they are green on the tip's CI (tests-ok, policy-sign, plugins social/tree/timeline/workflow, mock), which I read but did not re-run.
  • ARCH_DOCS_BASE=1e47e678 arch-docs — cannot diff against 1e47e678 without a git checkout here. I read the doc change instead: docs/subsystems/core-host.md#398 already owns this paragraph and gains one sentence for room-share: a legacy room's read recomputes the clamp target per line, so a long log reads in quadratic time #2766; arch-docs passed on the tip's CI.
  • A real-model end-to-end run: one call site in a pure read path.

What I checked, independently of the pull request's own narrative

  • The diff is exactly the change, no more: core/src/host/room-membership.js +6/−1 in linesFrom, its patches/ds-bot/files/ mirror (c2c669ee = c2c669ee, the same blob), the new test and its mirror (ecdca58b = ecdca58b), one changelog fragment and one doc sentence.
  • The defect holds on the base: at 1e47e678 the source is filter(line => (line.seq ?? 0) >= floorOf(room, actor)); floorOf calls clampTargetOf before its healthy-floor early return, and for a room with no seq that is (room.log ?? []).reduce(...) — a memo is only written on the clamped branch, so a legacy room whose floor is healthy reduces once per line. Measured: 32.8 s and 33.6 s per read at 20,000 lines, reduce calls=20000; at the tip 9 ms / 8 ms, reduce calls=1.
  • The fix is the one call moved out of the predicate, and nothing else: floorOf depends on the room and the reader, not on the line, and it is still called once per read, so the predicate, the room-share: a room whose stored joined floor is above its own seq shows nothing, and never heals #2754 memo, the rejoin drop and the one-time warning are the same code path.
  • Same visible window on base and tip: I read seven fixtures through both sources (legacy floor 0 / mid / above the log, modern floor mid / above seq, and both with an empty log) — linesVisibleTo (Alice, Bob, a non-participant), roomHistory and a second read agree line for line, and the clamped cases still warn exactly once. The only difference is that a room with an empty log and a stored floor above its clamp target now emits that same one-time warning, where base stayed silent only because floorOf never ran on an empty filter; no line changes, and this is the warning room-share: a room whose stored joined floor is above its own seq shows nothing, and never heals #2754 documents for exactly that state. Not a finding.

Security fence / display names: out of scope — the diff touches no delivery, social, fence, model-visible text or client bundle.

This report judges head 613cca30b83174571d7237786920f2971e403a07 only. It is a Verifier verdict, not merge authority: the risk tier is T3 (core/**), and the tip's enqueue-guard now needs this line to consider the queue.

@github-actions github-actions Bot added the pass label Nov 10, 2026
@Birfy
Birfy added this pull request to the merge queue Nov 10, 2026
Merged via the queue into ThinkFlowLab:main with commit 74d08c5 Nov 10, 2026
18 checks passed
@Birfy
Birfy deleted the dev/1-room-floor-once branch November 10, 2026 07:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

room-share: a legacy room's read recomputes the clamp target per line, so a long log reads in quadratic time

2 participants